MEMORANDUM October 31, 2018 


TO: Annie Wolfe 
Officer, Elementary Curriculum and Development 


FROM: Carla Stevens 
Assistant Superintendent, Research and Accountability 


SUBJECT: THE LITERACY EMPOWERED INITIATIVE: IMPACTS ON HISD HIGH 
SCHOOL STUDENTS’ PEFORMANCE IN CORE COURSES, 2017-2018 


The Literacy Empowered (LE) initiative provides support for all Houston Independent School 
District (HISD) high schools through expanded personalized instruction, the proficient use of 
PowerUp tools, and authentic literacy practices across all foundation courses. This evaluation 
report documented the impacts of the initiative on student performance in core high school 
courses based on the 2018 STAAR End-of-Course (EOC) exam results. The evaluation was 
guided by four questions and used a teacher survey, walkthrough visits, treatment effects, and 
multiple regression analyses to report on the implementation and impacts of the initiative. 


Key findings include: 

e The survey showed that 69.4 percent of secondary teacher respondents taught English, 
19.4 percent taught mathematics, 8.3 percent taught science, and 2.8 percent taught social 
studies. 

e On average, survey respondents tended to agree or somewhat agree with statements that 
were consistent with what was expected in their delivery of LE instruction. The weighted 
average was at least 3.50 of 5.0, except for statements regarding efficacy of preparation 
and LE implementation support, which had weighted averages of 3.14 and 3.21, 
respectively. 

e The highest weighted averages of the survey were with regards to LE instructional 
strategies and practices that involved talk, conversations, and asking questions with which 
respondents, on average, agreed or strongly agreed (weighted average of 4.00 and 4.07 of 
5.0). 

e Onaverage, survey respondents somewhat agreed or disagreed with statements regarding 
their use of the universal screener to assess and monitor students’ reading performance 
and growth and for placing them in flexible reading groups (weighted averages ranging from 
2.59 to 3.14 of 5.0). 

e Respondents recommended more LE strategies and activities and exposure to best LE 
practices that are tailored to core-content areas and the needs of a diverse HISD student 
and classroom population. 

e Aconsistently high percentage of LE students identified as gifted and talented (G/T) 
(94.2%—100%) and not at risk for school dropout (87.8%—96.8%) performed at or above the 
Approaches Grade Level standard on all 2018 STAAR EOC exams. 

e Treatment effects with regression adjustment (teffects ra) does not use matched treatment 
and control samples but estimates the counterfactual or missing data in the performance of 
treatment and non-treatment groups to determine performance differences and program 
effects. Results of treatment effects with regression adjustments analyses showed that a 
student selected at random, whose teacher did not participate in the LE professional 


development outperformed their peers whose teachers completed the PD on the 2018 
STAAR Algebra | EOC exam by 23 scale score points (ssp), Biology by 95 ssp, English | by 
122 ssp, English II by 48 ssp, and U.S. History by 233.9 ssp. 

e At-risk, English as a second language (ESL), special education, and gifted and talented 
(G/T) identification were the strongest predictors of LE students’ performance on the 2018 
STAAR Algebra |, Biology, English I, English Il, and U.S. History EOC exams. Only G/T 
identification was a positive predictor. 

e When students’ written composition scores were added to the regression model, the score 
predicted 60 and 58 percent of students’ performance on the STAAR English | and English 
Il EOC exams, respectively. 


Further distribution of this report is at your discretion. Should you have any further questions, 
please contact me at 713-556-6700. 


Attachment 


cc: Noelia Longoria 
Mechiel Rozas 
Jessica Chevalier 
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THE LITERACY EMPOWERED INITIATIVE: IMPACTS ON HISD 
HIGH SCHOOL STUDENTS’ PERFORMANCE IN CORE 
COURSES, 2017-2018 


Executive Summary 


The purpose of this evaluation was to determine the impact of the Houston Independent School District 
(HISD) Literacy Empowered (LE) initiative on the reading, literacy, and performance in core courses among 
high school students. Literacy Empowered is a districtwide initiative that provides support for high school 
students through the expansion of “personalized instruction through the proficient use of PowerUP tools 
and authentic literacy practices across all core foundation classes in preparation for postsecondary 
education and/or careers” (HISD, 2017a, p. VIII-55). 


Students who constituted the study sample were linked to their teachers who completed the LE professional 
development (PD) by using unique identifiers. The evaluation was guided by four questions and surveyed 
teachers who completed the professional development (PD) associated with the LE initiative. Students’ test 
data were analyzed, and treatment effect with regression adjustment was used to determine the impact of 
the initiative on students’ performance. Multiple regression was used to predict the performance of LE 
students in the four core subjects assessed using the State of Texas Assessments of Academic Readiness 
(STAAR) End-of-Course (EOC) exams and regressed on key demographic and educational variables. 


Key Highlights 


e The survey showed that 69.4 percent of secondary teacher respondents taught English, 19.4 percent 
taught mathematics, 8.3 percent taught science, and 2.8 percent taught social studies. 


e Survey respondents tended to agree or somewhat agree with statements that were consistent with 
what was expected in their delivery of LE instruction. The weighted average was at least 3.50 of 5.0, 
except for statements regarding efficacy of preparation and LE implementation support, which had 
weighted averages of 3.14 and 3.21, respectively. 


e The highest weighted averages of the survey were with regards to LE instructional strategies and 
practices that involved talk, conversations, and asking questions where respondents agreed or strongly 
agreed (4.00 and 4.07 of 5.0). 


e Survey respondents somewhat agreed or disagreed with statements regarding their use of the universal 
screener to assess and monitor students’ reading performance and growth and for placing them in 
flexible reading groups (weighted averages ranging from 2.59 to 3.14 of 5.0). 


e Survey respondents recommended more LE strategies and activities and exposure to best LE practices 
that are tailored to core-content areas and the needs of a diverse HISD student and classroom 
population. 


e Aconsistently high percentage of LE students identified as gifted and talented (G/T) (94.2%—100%) 
and not at risk for school dropout (87.8%—96.8%) performed at or above the Approaches Grade Level 
standard on Algebra |, Biology, English I, English Il, and U.S. History EOC exams. 
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e Treatment effects with regression adjustment (teffects ra) does not use matched treatment and control 
samples but estimates the counterfactual or missing data in the performance of treatment and non- 
treatment groups to determine performance differences and program effects. Results of the teffects ra 
analyses showed that a student selected at random, whose teacher did not participate in the LE 
professional development outperformed their peers whose teachers completed the PD on Algebra | by 
23 scale score points (ssp), Biology by 95 ssp, English | by 122 ssp, English II by 48 ssp, and U.S. 
History by 233.9 ssp. 


e §6At-risk, English as a second language (ESL), special education, and gifted and talented (G/T) 
identification were the strongest predictors of LE students’ performance on the 2018 STAAR Algebra I, 
Biology, English |, English Il, and U.S. History EOC exams. Only G/T identification was a positive 
predictor of STAAR EOC performance. 


e When students’ written composition scores were added to the regression model, the score predicted 
60 and 58 percent of students’ performance on the STAAR English | and English Il EOC exams, 
respectively. 


Recommendations 


e Although, the survey sample was limited, given the relatively low-weighted average assigned to 
respondents’ use of the universal screener, greater emphases need to be placed on the use of the 
universal screener and its importance in assessing and monitoring students’ reading performance and 
growth and for placing students in flexible learning groups. This should also facilitate targeted 
instructions that meet the reading and learning needs of high school students. 


e Survey respondents requested additional training to support the implementation of LE and exposure to 
the best practices and hands on approaches to teaching reading across the curriculum. Further, they 
requested more exposure to strategies that address students and classroom diversity, including ESL, 
special education, and G/T. 


e Survey respondents requested a more widespread involvement of teachers in the development of the 
master courses so that they meet the diverse needs of a diverse HISD student population. This may 
involve allowing teachers to review drafts and provide feedback on these courses before final approval. 


e Given the relatively low-weighted average on the selection of “Just Right” books and the significance 
of independent reading, students should be provided with the skills, time, and opportunities to select 
and read these books in and out of school. 


e Teachers should be provided adequate opportunities and time to implement LE strategies and activities 


or incorporate them into their course content areas. This may require the use of demonstration lessons 
and video exemplars for reference, when needed. 
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Introduction 


The Houston Independent School District (HISD) recognizes that strong reading, writing, communication, 
and thinking skills are the foundations for high school, college, and career success. The district commits to 
the inculcation of these skills through the Literacy Empowered initiative. The Literacy Empowered initiative 
is the literacy approach that provides support for all district high schools by expanding personalized 
instruction through the proficient use of PowerUp tools and authentic literacy practices across all core 
foundation classes in preparation for their postsecondary education and/or careers (HISD, 2017a, p. VIII- 
55). Literacy Empowered is designed to transform Tier 1 instruction in all HISD high schools by supporting 
teachers and leaders in the acquisition of the relevant pedagogy and tools that support 215' century student 
learning. It involves the utilization of existing computer and digital devices and core content master courses 
designed for students to personalize their learning pathways within state learning standards (HISD, 2017a). 
Literacy Empowered focuses on reading, writing, discourse using whole and small group learning, 
independent practice, and conferring among ninth through twelfth grade students in HISD during the 2017- 
2018 school year. 


In LE, students engage in reading self-selected and assigned texts beginning at their levels and expanding 
their systems of strategic action to meet the demands of increasingly complex text (HISD, 2017a). 
According to the 2017-2018 School Guidelines, “Instructional programs for grades 6—12 include time for 
students to read authentic, self-selected, and assigned reading text daily for at least 120 minutes within 
their four core foundation classes” (HISD, 201 7a, p. VIII-54). Students also engaged in authentic writing by 
discipline to expand and provide evidence of their textual understanding. 


To facilitate the implementation of the Literacy Empowered initiative, the district procured materials for the 
establishment of classroom libraries, with diverse Lexile-level texts and digital libraries with personalized 
texts across all four core-foundation academic disciplines. The district also secured comprehension tool 
kits for small and large group support, and document-based questions (DBQ) kits for social studies classes 
to support comprehension, analysis, and writing (HISD, 2017a). In preparation for the delivery of Literacy 
Empowered, school administrators, leaders, and teachers were exposed to professional development 
associated with the key elements of the initiative. 


The purpose of this evaluation was to determine the impact of Literacy Empowered on the reading and 
literacy attainment of ninth through to twelfth grade students in HISD. All high school students in HISD were 
eligible for participation in the program. The literature review, which follows, serves as a background to 
understanding the research on high school reading and literacy in the context of core-course instructions. 


Literature Review 


According to data from the National Center for Education Statistics (NCES), 37 percent of U.S. twelfth- 
grade students were reading at or above proficient level based on the 2015 National Assessment of 
Education Progress (McFarland, et al., 2017, May). Results of the State of Texas Assessments of Academic 
Readiness (STAAR) End-of-Course (EOC) exams indicated that 60 and 62 percent of HISD students 
performed at or above the Approaches Grade Level standard on the 2017 spring English | and English II 
exams, respectively (HISD, 2017b). This means that 40 and 38 percent, respectively, did not meet the 
standard the first time they took the exam. Literacy remains a critical issue in high school education. A 
report published in 2006 on the Standards for Middle and High School Literacy Coaches concluded “without 
targeted literacy instruction, many who graduate from high school will be ill-equipped for the demands of 
college or the new economy, relegated to remedial courses or dead-end jobs” (American Diploma Project 
cited in International Reading Association, 2006, p. 1). 
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The Center on Instruction made five recommendations for adolescent literacy instruction and provided study 
examples to substantiate each of these recommendations: (1) provide explicit instructions and supportive 
practice in the use of effective comprehension strategies throughout the school, (2) increase the amount 
and quality of open, sustained discussions on reading content, (3) set and maintain high standards for text, 
conversations, questions, and vocabulary, (4) increase students’ motivation and engagement with reading, 
and (5) teach essential knowledge so that all students master critical concepts (Torgesen, et al., 2007). 
Essentially, instruction, exposure to reading, and engagement with reading are crucial in literacy 
improvements beyond the elementary years. 


An action research study conducted by a high school teacher in Philadelphia, Pennsylvania demonstrated 
the use of key strategies that resulted in improved reading and reading engagement, appreciation for 
reading, and comprehension among honors students (Dickerson, 2015). These included (a) testing their 
reading levels prior to teaching, (b) increasing time for silent sustained or independent reading, (c) providing 
a variety of texts and allowing students to select their own texts, (d) getting student to write about and 
discuss their readings using notebooks, and (e) modeling reading by reading aloud and reading along with 
students. Dickerson (2015) also administered a student survey at the end of each quarter to determine 
students’ perceptions and experiences. The survey showed positive feedback on the strategies used and 
their impacts on reading and comprehension. 


As part of a publication volume dedicated to “Reading: The core skills,” Allington and Gabriel (2012), 
identified six elements for reading that children should experience every day: (1) every child reads 
something he or she chooses; (2) every child reads accurately; (3) every child reads something he or she 
understands; (4) every child writes about something personally meaningful; (5) every child talks with peers 
about reading and writing; and, (6) every child listens to an adult read fluently. They contend that most of 
the classrooms they observed lacked these six elements. Allington and Gabriel (2012) called for the 
elimination of worksheets and workbooks and to use the resources to procure books, using the time saved 
for self-selecting readings, self-selected writing, literacy conversations, and read alouds; and to ban test 
preparation activities and materials during the school day. There are no studies that demonstrate improved 
reading or test performance because of student engagements in test preparation (Guthrie cited in Allington 
& Gabriel, 2012). Similar elements for reading were identified in the work of Allyn (2012) and Scherer 
(2012). These included offering students a range of reading materials, providing time for dialogue on 
readings, providing readers with a tool kit, letting readers read at their comfort levels, discussing books 
deeply and including students’ reactions to their readings, allowing browsing and rereading, teaching 
students to curate their own reading lives, and allowing reading to be a joyful experience. 


The implementation of these strategies and activities, however, are dependent on teachers and their 
perspectives of struggling readers. Moreau’s (2014) survey of 39 middle school teachers across three 
different school districts found that (1) respondents believe that middle school students should already be 
competent grade-level readers, and (2) they did not believe that it was their job to teach specific reading 
skills in content area classes since they were restricted by a lack of knowledge and time. Clearly, changing 
those perceptions may be key in addressing middle and high school reading difficulties. 


Similarly, based on science and social studies classroom observations, Ness (2009) reported that only 
three percent of 2,400 instructional minutes observed were devoted to reading comprehension. On inquiry, 
teachers did not feel that they were responsible or qualified to provide explicit instruction on reading 
comprehension. They highlighted the pressure associated with covering content in preparation for 
standardized state assessments as hindrances to providing reading instruction (Ness, 2009). 
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In summary, only reading will improve reading, more of it, with books students love and choose, while 
allowing them to write about and discuss their readings, monitoring their progress, listening to others read 
fluently, and eliminate worksheets, workbooks, and test preparation time to free resources and time for 
reading and discussion of reading materials. The evaluation focused on four key questions: 


1. How did teachers perceive and experience the implementation of Literacy Empowered (LE)? 
2. How did the observed lessons reflect key aspects of the LE initiative? 
3. What is the demographic and educational composition of students in the evaluation sample? 


4. What was the effect of LE on the students’ performance on the 2018 state assessments? 


Method 


This evaluation is a quasi-experimental study designed to determine the impact of the LE initiative on 
student performance on the State of Texas Assessments of Academic Readiness (STAAR) End-of-Course 
(EOC) exams. Quasi-experimental studies use observational data to make inferential statements about the 
population of students using a representative sample of students. It is considered an alternative to the 
random control trials (RCT) in which subjects are randomly selected into control and treatment groups as 
a precondition for making inferential statements about the population from which the sample was drawn. 


Principals, school administrators, and leaders, and teachers were exposed to three to 24 credit hours of 
professional development (PD) in preparation for the implementation of LE. The PD was available to 
teachers and other school personnel across the curriculum and included core-subject teachers of Algebra 
|, Biology, English, and U.S. History. All students whose core foundation teachers completed the PD were 
included in the analysis. The comparison group consisted of students whose teachers did not participate in 
the PD. Students were grouped by STAAR EOC exams. 


A web-based survey was administered to the teachers who participated in the LE professional development 
to determine how they perceived and experienced the professional development, and the implementation 
associated with LE. The survey was developed in collaboration with two teacher development specialists 
who were involved in the implementation and supervision of the initiative. They ensured the survey was 
valid and tested before final submission to participants. The survey was disseminated to 121 teachers using 
SurveyMonkey™, which is a web-based data collection and analysis portal. 


A random sample of ten classes, derived from a list of teachers who completed the LE professional 
development, was selected for walkthroughs. Random numbers were assigned to each of the teachers and 
the sample selected. Requests for permission to visit the school and conduct these walkthroughs were 
emailed to school principals with a reminder two weeks later. The purpose was to observe lesson delivery, 
the use of LE strategies, and the presence of artifacts in the classrooms as evidence that these strategies 
were utilized or being utilized. Two schools consented to the visits. The visits were conducted with the 
assistance of a teacher development specialist (TDS) for reliability. 


Data Collection 


Teachers’ IDs from the eLearning databases for the LE initiative were used to link these teachers to their 
students. Student identification numbers (IDs), in turn, were used to link students to their STAAR EOC 
exam results and the demographic and educational data from the HISD Public Education Information 
Management System (PEIMS) using Microsoft Access. Data were exported, cleaned, and formatted in 
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Microsoft Excel spreadsheet and analyzed in Stata. Stata is a statistical software for the analysis of 
observational data. 


The web-based survey questionnaire was disseminated to LE teachers using their email addresses. A 
reminder was sent out fifteen days later. Thirty-six or 29.8 percent of teachers responded to the survey. 
The response rate is consistent with observed trends for online surveys (See Saldivar, 2012; Hamilton as 
cited in Saldivar, 2012; HISD, 2015). The survey consisted of descriptive or Likert scale questions. The 
Likert scale questions measured respondents’ degree of agreement or disagreement with statements on 
key LE components on a scale of 1—5 (1 = strongly disagree, 2 = disagree, 3 = somewhat agree, 4 = agree, 
and 5 = strongly agree). 


An ethnographic approach using field notes was used to collect data on lesson deliveries during the 
walkthrough. Photos were also taken of lesson artifacts and other related components of the initiative 
present in the classrooms. Principals or assistant principals were debriefed after the visits. 


Data Analysis 


The data analyses involved the description of LE students’ performance on the state assessments, 
summary of walkthroughs, measurement of LE treatment effects, and survey analyses of teachers who 
completed the PD. Teacher PD was used because these teachers would have acquired the skills and 
protocols necessary to implement the initiative’s key components with fidelity: reading, writing, discourse 
using whole group and small group learning, independent practice, and conferring. It was anticipated that 
students whose teachers completed the PD would outperform their peers whose teachers did not complete 
the PD on the STAAR assessment, assuming the initiative was implemented with fidelity. This argument is 
consistent with current standards in the evaluation of educational programs and initiatives that require 
reporting program effects and effects sizes (Yoon, Duncan, Lee, Scarloss, & Shapley, 2009). U.S. 
Department of Education Institute for Education Sciences (IES) also designated an effect size in education 
of .25 as “substantively important” (IES, 2017, p. 22) 


Stata treatment effect with regression adjustment (teffects ra) was used to determine the impacts or effects 
of the LE initiative on the STAAR EOC exam performance of students whose teachers participated in LE 
professional development prior to the delivery of the initiatives’ key components. Teffects ra allows for the 
estimation of quasi-experimental casual effects from observational data. It estimates average treatment 
effects (ATE), among others, using robust or doubly robust estimators. Teffects does not use matched 
samples to determine program effects. Teffects ra determines the differences in the outcomes or 
performance of LE students and non-LE students by estimating the counterfactual or missing data for these 
LE and non-LE students. That is, how would LE students perform on the STAAR OEC exams if their 
teachers were not exposed to the LE PD, and how would non-LE students perform, if their teachers were 
exposed to the same PD. The impact is measured by the difference in the average outcome or performance 
of the treatment and non-treatment or comparison groups. Teffects have the advantage of regressing the 
attributes on either the treatment or outcomes without adversely affecting the error term or the selection 
bias (Cameron & Trivedi, 2005; Huber, 2015). In this case, it is regressed on the outcome and none of the 
attributes are related to the treatment. Regression adjustment (ra) estimators model the outcome to account 
for the non-random treatment of the assignment into treatment and non-treatment groups (Huber, 2015). 


Survey data were summarized in SurveyMonkey™, exported to Microsoft Excel and presented as 
descriptive data in tables and charts. A summary analysis of the classroom observations is included to 
reflect the outcomes of the walkthroughs. These outcomes are not representative of delivery of the initiative 
districtwide and should be interpreted with caution. 
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Limitations 


Only two schools consented to the walkthrough making it difficult to determine the extent to which LE 
strategies were being implemented with fidelity. Care should be taken in the interpretation of the 
walkthrough data and information since the observations may not be representative of the entire 
delivery of the initiative. 


Literacy Empowered is a districtwide initiative, however schools and teachers appeared to have been 
self-selected for participation into the initiative. Students whose teachers did not participate are used 
as proxies for a control group in this quasi-experimental study. Those non-participating teachers, 
however, may not have had similar desires and motivation to participate in the initiative. 


Results 


How did teachers perceive and experience the implementation of Literacy Empowered (LE)? 


Results of the LE teachers’ survey are presented in charts and tables. The educational and instructional 
configurations, the instructional and implementation experiences of LE teachers, self-efficacy, and 
instructional supports as well as teachers’ perceptions of their students’ LE activities are presented in this 
section of the report. Figure 1 shows the grade assignments for teachers who responded to the survey. 
Table A1, Appendix A, p. 21 provides details. 


Figure 1. Percentage Distribution of Grade Assignments for LE Survey Respondents, 2017-2018 (n 
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Most survey respondents taught ninth (41.7%) and tenth (27.8%) grade students. 


According to Figure 1, 8.3 percent of respondents were twelfth-grade teachers. It also shows that 11.1 
percent of respondents taught either eleventh or other configurations, each. Other assignments meant 
multiple grades. 


Figure 2 displays the percentage distribution of the years of teaching experience of LE survey respondents. 
Table A2, Appendix A, p. 21 provides details. 
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Figure 2. Percentage Distribution of LE Survey Respondents’ Years of Teaching Experience, 2017- 
2018 (n = 36) 
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Years of Teaching 


e Most survey respondents (27.8%) had twenty or more years of teaching experience, followed by 22.2 
percent with 6-10 years of teaching experience. 


e Based on Figure 2, 16.7 percent of the survey respondents had 11-15 years teaching experience. 


e Only 8.3 percent of the survey respondents had 0-2 years of teaching experience. 


Figure 3 shows the teaching roles of LE survey respondents. Table A3, Appendix A, p. 21 provides details. 


Figure 3. Percentage Distribution of Teaching Roles of LE Survey Respondents, 2017-2018 (n = 
36) 
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Teaching Roles 


e The majority (91.7%) of the LE survey respondents were core-content teachers. 


e According to Figure 3, 5.6 percent of LE survey respondents were interventionists and 2.8 percent 
were co-teachers. 


Figure 4 shows the core-content areas taught by LE survey respondents. Table 4, Appendix A, p. 21 
provides details. 
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Figure 4. Core-Content Areas Taught by LE Survey Respondents, 2017-2018 (n = 36) 
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Core Content Teaching Area 
e Most LE survey respondents (69.4%) taught English as a core-content area. 


e According to Figure 4, 19.4 percent of the LE survey respondents taught mathematics, 8.3 percent 
taught science, and 2.8 percent were social studies teachers. 


Figure 5 displays the number of instructional minutes allocated for the delivery of LE as reported by survey 
respondents. Details are provided in Table A5, Appendix A, p. 22. 


Figure 5. Percentage Distribution of the Approximate Number of LE Respondents Instructional 
Minutes, 2017-2018 9 (n = 36) 
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Literacy Instruction 


e Most respondents (47.2%) had less than a 50-minute timeframe in which to implement or incorporate 
the LE initiative into their instructions. This was followed by respondents who had 90 minutes (25%) in 
which to incorporate LE instructions and strategies. 


e According to Figure 5, 13.9 of respondents claimed to have had either 75 minutes or more than 90 
minutes, each, within which to implement the LE initiative. 


Figure 6 shows the LE language of instruction distributed by the percentage of survey respondents in the 
study. Table A6, Appendix A, p. 22 provides details. 
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Figure 6. Percentage Distribution of Primary Language of Instruction Among LE Survey 
Respondents, 2017-2018 (n = 36) 
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e ~All the LE survey respondents instructed in either English (97.2%) or Chinese Mandarin (2.8%). 


Figure 7 shows the participation rate for teachers who were engaged in the LE professional development. 
Table A7, Appendix A, p. 22 provides details. 


Figure 7. Professional Development Participation Rates for LE Survey Respondents, 2017-2018 (n 
= 29) 
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e Most respondents, between 82.8 and 89.7 percent, appeared to have attended at least one of the four- 
day literacy empowered training. 


e Between 10.3 and 17.2 percent of respondents did not attend at least one of the LE training day 
sessions. 


Figure 8 displays the weighted averages of survey respondents’ agreement or disagreement with their 
teaching efficacy, resources, and instructional practices associated with the LE initiative, on a scale of 1 to 
5, using ten related statements. Table A8, Appendix A, p. 23 provides details. 
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Figure 8. Efficacy, LE Resources, and Instructional Practices of Survey Respondents, 2017-2018 
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area. 


On average, survey respondents appear to agree or somewhat agree (weighted average of 3.55 of 5.0) 
that, compared to one year ago, they had a deeper understanding of how to effectively incorporate 
literacy instruction into their core-content area. 


On average, respondents agreed or somewhat agreed that the digital and print reading materials and 
resources available to their classrooms were useful (weighted average of 3.52 of 5.0) and of high quality 
(weighted average of 3.72 of 5.0). 


On average, survey respondents agreed or somewhat agreed (weighted average 3.76 of 5.0) that they 
facilitated the consumption of a high volume of text that included images, graphs, and charts. They 
generally agreed (weighted average 3.69 of 5.0) that their students wrote for authentic purposes with 
varied content, audiences, and modes. 


Respondents appeared to agree or somewhat agree (weighted average of 3.10 of 5.0) on whether 
students in their classrooms independently selected “Just Right” books or texts for independent 
reading, and whether they read various books and texts on their independent reading levels during 
daily independent reading time in their classrooms (weighted average of 3.21 of 5.0). 


Respondents either disagreed or somewhat agreed that they used the universal screener to assess 
student reading levels (weighted average of 3.10 of 5.0), monitor student reading growth (weighted 
average of 3.14 of 5.0), and form flexible reading groups (weighted average of 2.59 of 5.0). 


Figure 9 shows the weighted averages of survey respondents’ agreement or disagreement with LE 
teaching and learning practices based on a scale of 1 to 5 using seven related statements. Table A9, 
Appendix A, p. 23 provides details. 
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Figure 9. Teaching and Student Learning Practices of LE Survey Respondents, 2017-2018 (n = 29) 
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e Survey respondents agreed or somewhat agreed (weighted average 3.52 of 5.0) that their students 
wrote for authentic purposes using varied content, audiences, and modes but on average, they 
somewhat agreed (weighted average of 3.14 of 5.0) that their students were able to archive written 
work with portfolios. 


e Respondents agreed or somewhat agreed (weighted average of 3.48 of 5.0) that their students 
analyzed their work using rubrics and that they utilize writing as a learning tool (weighted average of 
3.68 of 5.0). 


e Respondents agreed or strongly agreed (weighted average of 4.07 of 5.0) that they used open-ended 
questions and talked to facilitate discourse in their classrooms and that they employed structured 
conversations that focused on content and cognition in their instruction (weighted average of 4.00 of 
5.0). 


e Survey respondents agreed or strongly agreed (weighted average of 3.69 of 5.0) that dialogic discourse 
took precedence over monologic discourse in their classrooms. 


Figure 10 shows, on a scale of 1 to 5, the weighted averages of LE survey respondents’ agreement or 
disagreement with two LE instructional support statements. Table A10, Appendix A, p. 24 provides details. 
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Figure 10. Instructional Support for LE Survey Respondents, 2017-2018 (n = 29) 


The information | learned during the Literacy The leaders on my campus provide valuable support 
Empowered summer training (S) prepared me for the and information about best literacy practices in the 
implementation of Literacy Empowered in my content areas. 
classroom. 


e On average, teacher respondents agreed or somewhat agreed (weighted average of 3.31 of 5.0) that 
the information they learned during the LE summer training prepared them for the LE implementation 
in their classrooms. 


e Onaverage, teacher respondents either agreed or somewhat agreed (weighted average of 3.14 of 5.0) 
that leaders on their campuses provided valuable support and information about best LE practices in 
their content areas. 


Survey respondents were asked to indicate their agreement or disagreement with statements related to 
HISD secondary core-content master courses and the implementation of the LE initiative. Figure 11 
displays the responses. Table A11, Appendix A, p. 24 provides details. 


Figure 11. LE Survey Responses on Structure and Framework of HISD Secondary Core-Content 
Master Courses, 2017-2018 (n = 29) 
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e On average, respondents agreed or somewhat agreed that the structure and framework of the HISD 
secondary core-content master courses facilitated the ease of planning for their implementation of 
Literacy Empowered (weighted average of 3.36 of 5.0). 
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e On average, respondents agreed or somewhat agreed that the instructional strategies outlined in the 
HISD secondary core-content master courses supported their LE instructional planning and delivery 
(weighted average of 3.43 of 5.0). 


Survey respondents were also asked two open-ended questions: (1) What additional professional 
development opportunities would you like to receive to further support Literacy Empowered 
implementation? and (2) What additional details/information would you like to see in the HISD master 
courses to further support LE implementation? Respondents were also asked to provide any additional 
information that may not have been addressed in the survey. Responses are summarized in the two 
sections which follow. 


Additional Professional Development Proposed 


There were 29 responses to the related question on proposing additional LE-related professional 
development. Respondents wanted exposure to more best practices and more time to implement the 
Literacy Empowered professional development. They also wanted more Literacy Empowered training and 
resources to enhance implementation. Details are provided in Table A12, Appendix A, p. 25. 


Improvement in HISD Master Courses 


With respect to improvements to the HISD master courses for the delivery of LE, 19 responses were 
provided. These responses are detailed in Table A13, Appendix A, p. 26. 


e Teacher respondents wanted exposure to more best practices, including current strategies that are 
successful to further support literacy implementation. 


e One respondent who taught at an Achieve 180 school, found it hard to devote time to review the master 
courses and adjust them to the needs of his/her scholars. 


e Respondents wanted more specific needs-based professional development training, particularly 
geared toward the needs of Early College students or “reality pedagogy” for English language learners, 
special education students, and students identified as gifted and talented. 


e Respondents also wanted a team approach to the development of the master courses to reduce their 
perceived repetitiveness and to increase the variety of strategies for delivering these master courses. 


How did the observed lessons reflect key aspects of the LE initiative? 


Two English language arts (ELA) lessons were observed in two high schools. Lesson One used the three- 
big-questions strategy to assist students in comprehending a passage on the permanent closure of the 
Ringling Brothers Circus based on allegations of animal cruelty. The lesson objective and agenda with time- 
on-task were clearly written on the dry-erase board (Figure B1, p. 27). 


Observation details of Lesson One are in Appendix B, p. 27. Students were able, in small group 
discussions, to identify the key points of the article by answering the three big questions (1) What surprised 
me? (2) What did the author expect me to already know? and (3) What changed, challenged, or confirmed 
what | thought | knew (Figure B2, p. 28)? Students wrote their responses on flipchart paper and posted 
them on the classroom walls (Figure B3, p. 28). The teacher modeled responses to those big questions, 
using a whole-class approach, before dividing the class into small groups. Students read the article aloud 
or asked their peers to read aloud before beginning the exercise. Students appeared to understand the 
purpose and objective of the article based on their responses, discussion, and their charts. An inquiry into 
students’ previous knowledge and familiarity with the circus would have established some context and 
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experience for a more meaningful treatment of the article. Students’ own point of view could have been 
sought and served as the basis for later writing on the topic or issues dear to them. 


Students’ writing folders (Figure B4, p. 29) and “Saddleback” and “Just Right” books were neatly and 
prominently arranged on classroom shelves. A review of five of the folders showed students’ attempts at 
writing but did not reflect the writing process sequenced from draft to publication pieces. Parental forms}, 
as seen in the folders, were unsigned. This seemed to be consistent with survey responses on written 
portfolios statement in Figure 9, p.12 in which teachers tended to somewhat agree (3.14 of 5.0) on their 
students’ ability to archive written work using portfolios. Notwithstanding, the use of whole calls as a model 
for answering the three big questions, the use of the three big questions in small groups, and the read 
alouds were all key aspects of the LE initiative. 


Lesson Two (Appendix B, p. 30) also had clearly a written objective and agenda posted in the classroom. 
Students were seated in table groups as the teacher used three prompts posted on flipchart paper to assist 
students in writing their own creation myths. The prompts were (1) Must have, (2) Might have, and (3) Will 
not have (Figure B5, p. 30). Students were quizzed to complete the prompts based on their apparent prior 
knowledge on writing. 


While the learning objective focused on comprehension and expanded reading skills, students were given 
a writing activity with eight guiding questions. In addition, students were given eight questions, listed on the 
classroom whiteboard, to guide the development of their own creation myth. Figure B6, (p. 31) displays 
the questions. 


It appeared inconclusive what the class assignment was designed to accomplish, considering the LE 
components. The class assignment itself seemed to require more than the allocated one-hour class for 
completion. Eventually, students were told that the task would be continued in the next class if they ran out 
of time. Further, as mentioned, the lesson objective appeared to be incompatible with the class assignment 
unless this was the continuation of a prior lesson but that was not communicated. The use of writing 
prompts, guiding questions, and small or table groups and the whole-class approach to developing the 
prompts were consistent with key LE components. 


What is the demographic and educational composition of students in the LE evaluation sample? 


Students’ STAAR EOC exam results were used to determine the effect of the Literacy Empowered initiative 
on the performance of students whose teachers completed the professional development. It was also used 
to predict students’ performance on the 2018 STAAR EOC Algebra I, Biology, English |, English II, and U.S. 
History exam results. 


Table C1, Appendix C, (p. 32) shows the sample of students (10,277) whose teachers completed the LE 
professional development and 26,807 students whose teacher did not participate in the PD. The district 
data were used for context. 


e There was a substantially lower proportion of ESL (23.3% vs 68.5%), Asian (2.5% vs 4.1%), White 
(4.3% vs 8.7%), and special education (3.6% vs 7.2%) students in the treatment sample than there 
were in the general student population in HISD. 


1 Parental forms are paper forms that are to be signed by parents as evidence that they had seen and 
reviewed their student’s writing portfolio. These were inserted into the portfolios. 
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What was the effect of LE on students’ performance on the 2018 state assessments? 


Table C2, Appendix C, (p. 32) displays the percentage of LE students who performed at or above the 
Approaches Grade Level standard on the STAAR EOC exams. 


A consistently high percentage of LE students identified as gifted and talented (G/T) (94.2% — 100%) 
and students who were not at risk for school dropout (87.8% - 96.8%) performed at or above the 
Approaches Grade Level standard on all EOC exams. 


Less than 50 percent of special education LE students performed at or above the Approaches Grade 
Level standard on the STAAR EOC exams, except U.S. History (54.3%). 


Less than 50 percent of English as a second language (ESL) LE students performed at or above the 
Approaches Grade Level standard on the STAAR EOC English |, English II, and U.S. History. 


Between 71.1 percent and 76.2 percent of at-risk LE students performed at or above the Approaches 
Grade Level standard on Algebra I, Biology, and U.S. History EOC exams. Less than 50 percent of at- 
risk LE students performed at or above the Approaches Grade Level Standard on STAAR English | 
(44.3%) and English II (49.2%) EOC exams. 


Table C3 to Table C7 in Appendix C, (pp. 33-34) show the average treatment effects (ATE) of the LE 
initiative on the STAAR EOC Algebra |, Biology, English I, English Il, and U.S. History exam results of 
students whose teachers completed the LE professional development. 


Treatment effects with regression adjustment (teffects ra) does not use matched samples but estimates 
the counterfactual or missing data in the performance of treatment and non-treatment groups to 
determine performance differences and program effects. Based on the teffects ra results, a high school 
student, selected at random, whose teacher was not exposed to the LE professional development, 
would have outperformed his/her peers whose teachers completed the professional development on 
the STAAR EOC Algebra | by 23.7 scale score points (ssp), Biology by 95 ssp, English | by 122.2 ssp, 
English II by 48 ssp and U.S. History by 233.9 ssp. The differences were all statistically significant (p < 
.001) (two-tailed). 


Table C8 to Table C12, Appendix C, (pp. 34-35) show selected predictors of LE students’ performance 
on the STAAR EOC Algebra |, Biology, English I, English Il, and U.S. History exam results, respectively. 


Being at-risk for school dropout (at-risk) (23%), gifted and talented (G/T) identification (19%), special 
education (14%), English as a second language (ESL) (12%), and being male (7%) were all significant 
predictors of LE students’ performance on the STAAR EOC Algebra | exam (p < .001) (two-tailed). 
Overall, however, the model predicted only 17 percent of the Algebra | performance. Being at-risk and 
G/T identification were the strongest predictors, with the latter being the only positive predictor (Table 
C8, Appendix C, p. 34). 


Being at-risk (32%), ESL (22%), G/T identification (21%), and special education (12%) were significant 
predictors of LE students’ performance on the STAAR EOC Biology exam (p < .001) (two-tailed). 
Overall, the model predicted 30% of students’ performance in Biology. Being at-risk and ESL were the 
strongest predictors. G/T identification was the only positive predictor (Table C9, Appendix C, p. 34). 


ESL (37%), at-risk (26%), G/T identification (22%), male (13%), and special education (12%) were 
statistically significant predictors of LE students’ performance on the STAAR English | exam for Model 
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| (p < .001) (two-tailed). Being ESL and at-risk were the strongest predictors. G/T identification was the 
only positive predictor. Overall, the model predicted 42 percent of students’ performance on the English 
| EOC exam (Table C10, Appendix C, p. 35). Writing composition was added to Model 2. Overall, Model 
2 predicted 70 percent of the performance on English | while writing alone positively predicted 60 
percent of the English | score and reduced the predictability of ESL, at-risk, and G/T, substantially. 


e All seven selected predictors, ESL (39%), at-risk (28%), G/T (27%), special education (11%), Career 
and Technical Education (CTE) (9%), economically-disadvantaged (9%), and gender (5%) were 
statistically significant (p < .001) for LE student’s performance on the STAAR English II exam. Being 
ESL and at-risk were the strongest predictors. Overall, the model predicted 60 percent of students’ 
performance on the English Il EOC exam (Table C11, Appendix C, p. 35). Overall, Model 2 predicted 
79 percent of LE students’ English II performance. Writing composition positively predicted 58 percent 
of that score while substantially reducing the predictability of ESL, at risk, and G/T. 


e ESL (37%), at-risk (23%), G/T identification (18%), and special education (14%) were statistically 
significant predictors of LE students’ performance on the STAAR U.S. History EOC exams (p Ss .05). 
ESL and at-risk were the strongest predictors of U.S. History performance. Overall. the model predicted 
30 percent of LE students’ performance on the U.S. History results (Table C12, Appendix C, p. 35). 


Discussion 


This evaluation sought to answer four questions to shed light on the implementation of the Literacy 
Empowered initiative and its effect on students’ performance. To answer these questions, core-content 
teachers who participated in LE professional development prior to implementation were surveyed to 
determine their related perceptions and experiences. Students’ performance on STAAR EOC assessments 
for Algebra |, Biology, English I, English Il, and U.S. History were analyzed using treatment effect with 
regression adjustment (teffects ra) on key demographic and educational variables. In addition, students’ 
performances were regressed on key variables as predictors of performance in the EOC assessments, and 
walkthrough visits were conducted to observe the implementation of the LE initiative. 


Teacher respondents tended to agree with statements that were consistent with what was expected in their 
delivery of LE instruction. Their weighted averages for these statements were at least 3.50 on a 5.0 Likert 
Scale. Respondents appeared to agree or somewhat agree with statements regarding the preparation for 
and support associated with the implementation of the LE Initiative. In this case, their weighted averages 
ranged from 3.14 to 3.21. Overall, respondents had their highest average weights for statements about 
instructional practices that involved talk, conversations, and asking questions (4.00 and 4.07). 


Respondents were varied in their assessment of their LE student practices. They appeared to agree more 
with statements about students’ writing practices (writing for authentic purposes and using writing as a 
learning tool) weighted averages for these statements did not exceed 3.68 of 5.0. Respondents appeared 
to be less confident about their students’ ability to select “Just Right’ books independently or to read texts 
on their independent levels during independent reading (3.10 and 3.21, respectively). This is critical since 
effective practice for high school reading recommends increased time for independent reading and that 
students should have choice books, self-selected for independent reading (Allington & Gabriel, 2012; Allyn, 
2012; Dickerson, 2015; Scherer, 2012). 


Overall, respondents barely agreed with statements related to their use of the universal screener to assess 
and monitor students’ reading and reading growth and placing them into flexible reading groups. The 
weighted averages for these statements ranged from 2.59 to 3.14. Failure to adequately assess and monitor 
students’ reading would pose a challenge for placing them into those flexible reading groups and adequately 
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meeting students’ reading needs. Overall, this statement had the lowest average rating of the survey at 
2.59. Dickerson (2015) recommended the reading assessment of all students prior to teaching as one of 
the key strategies in effective high school reading instruction. Research on teaching reading and 
comprehension in core-content areas indicate that teachers do not feel that they have the time and 
opportunity to teach reading skills because of the emphasis on passing standardized tests (Moreau, 2014; 
Ness, 2009). Respondents suggested more class time to implement LE and requested that administrators 
provide opportunity to implement LE initiatives “instead of doing STAAR prep.” Further, the survey showed 
that most respondents had less than 50 minutes per class in which to teach content and incorporate LE. 
The HISD School Guidelines, 2017-2018 recommended 120 minutes for literacy instruction across sixth- 
to twelfth-grade core foundation classes (HISD, 2017 a). It is unknown how this is to be parsed out among 
these core foundation classes. 


Most student groups in the sample appeared to have performed at or above Approaches Grade Level 
standard in Algebra |, Biology, and U. History EOC exams. The performance for English | and English II 
were varied, ranging from a high of 87.8 and 96.3 percent, respectively for non-at-risk students and a low 
15.2 and 10.1 percent, respectively for English as a second language students. It is the expectation that LE 
initiative strategies and activities would most likely be used in English | and English II courses but as will 
be shown later there were other factors that could have influenced students’ performance in the core 
content area. 


Results of the Teffects ra showed that a student, whose teacher was in the control group, selected at 
random from the population of high school students would have scored better on the 2018 STAAR EOC 
Algebra |, Biology, English |, English Il, and U.S. History assessments than their peers whose teachers 
completed the LE initiative PD. Again, teffects ra does not use matched groups to determine program 
effects. Using the counterfactual or the missing data estimated by the regression adjustment (ra), teffects 
computed the average performance differences between the treatment and control groups by EOC exams 
results. Again, failure to adequately assess and monitor students’ reading performance and growth may 
mean that respondents were not able to effectively identify students’ reading needs and to tailor instruction 
or utilize appropriate strategies to meet the needs of these students. This is further supported by the failure 
of respondents to agree that their students can select “Just Right” books independently. The research 
shows that independent reading and students’ independent selection of choice books were key elements 
in the improvement of high school reading (Allington & Gabriel, 2012; Dickerson, 2015). Additionally, 
teachers’ low rating on their LE preparation and support does suggest that they believed that may not have 
had all they needed to effectively implement the LE initiative. Respondents requested more LE training, 
more exposure to best LE instructional practices, and more resources to improve implementation. 


There are other factors that impinge on LE students’ performance on the STAAR EOC tests. English as a 
second language (ESL), at risk for school dropout, and G/T identification were the strongest statistically 
significant predictors of students’ performance on all five STAAR EOC tests for 2018. G/T was the only 
positive predictor. Writing composition predicted a substantial part of English | and English II performance 
for LE students in Model 2. Addressing the effects of these predictors on students’ reading are critical to 
improve the effectiveness of the LE initiative. 


Respondents suggested a team approach to the development of master courses to increase the length of 
the lessons and reduce their formulaic and repetitive nature. They also called for what they described as 
“realistic pedagogy” that addressed classroom diversities like early college, English language learners 
(ELL), special education, G/T, and low-performing, non-special-education students. Respondents also 
believed that contextually, the master courses should be flexible enough to meet the needs of Achieve 180 
students and classrooms. 


HISD Research and Accountability 18 


LITERACY EMPOWERED REPORT, 2017-2018 


Recommendations 


e Although, the survey sample was limited, given the relatively low-weighted average assigned to 
respondents’ use of the universal screener, greater emphases need to be placed on the use of the 
universal screener and its importance in assessing and monitoring students’ reading performance and 
growth and for placing students in flexible learning groups. This should also facilitate targeted 
instructions that meet the reading and learning needs of high school students. 


e Survey respondents requested additional training to support the implementation of LE and exposure to 
the best practices and hands on approaches to teaching reading across the curriculum. Further, they 
requested more exposure to strategies that address students and classroom diversity, including ESL, 
special education, and G/T. 


e Survey respondents requested a more widespread involvement of teachers in the development of the 
master courses so that they meet the diverse needs of a diverse HISD student population. This may 
involve allowing teachers to review drafts and provide feedback on these courses before final approval. 


e Given the relatively low-weighted average on the selection of “Just Right” books and the significance 
of independent reading, students should be provided with the skills, time, and opportunities to select 
and read these books in and out of school. 


e Teachers should be provided adequate opportunities and time to implement LE strategies and activities 
or incorporate them into their course content areas. This may require the use of demonstration lessons 
and video exemplars for reference, when needed. 
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Appendix A 
Table A1. Grade Level Assignment of Literacy Empowered Survey Respondents, HISD, 2017- 
2018 
Grade level % n 
Ninth 41.7 15. 
Tenth 27.8 10 
Eleventh 14.1. 4 
Twelfth 8.3 3 
Other (please specify) 11.4 4 
Total 36 
No responses 0 


Table A2. Teaching Experience of HISD Literacy Empowered Survey Respondents, 2017-2018 


Years % n 
0-2 years 8.3 3 
3-5 years 13.9 5 
6-10 years 22.2 8 
11-15 years 16.7 6 
16 -20 years 11.1. 4 
20+ years 27.8 10 
Total 36 
No Responses 0 


Table A3. Teaching Configurations of HISD Literacy Empowered Survey Respondents, 2017- 


2018 
Teaching Configuration 
Core-content teacher 91.7 33 
Co-teacher 2.8 1 
Interventionist 5.6 2 
Other (please specify) 0.0 0 
Total 36 
No Responses 0 


Table A4. Frequency Distribution of Core Courses Taught by HISD Literacy Empowered Survey 
Respondents, 2017-2018 
n 


Core Course % 
English 69.4 25 
Mathematics 19.4 7 
Social Studies 2.8 1. 
Science 8.3 3 
Total 36 
No Responses 0 
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Table A5. Daily Literacy Instructional Time Among Literacy Empowered Survey Respondents in 


HISD, 2017-2018 


Instruction Time % n 
Less than 50 minutes 47.2 17 
75 minutes 13.9 5 
90 minutes 25.0 9 
More than 90 minutes 13.9 5 
Total 36 
No Responses 0 


Table A6. Survey Respondents’ Language of Instruction for Literacy Empowered in HISD, 2017- 


2018 

Language of Instruction % n 
English 97.2 35 
Spanish 0.0 0 
Chinese Mandarin 2.8 1 
American Sign Language 0.0 0) 
Other (please specify) 0.0 0 
Total 36 
No Responses 0 


Table A7. HISD Literacy Empowered Professional Development Participation of Survey 


Respondents, 2017-2018 


7 Yes No 
Professional Development % = % = Total 
| attended the Literacy Empowered training, Day 1. 89.7 26 10.3 3 29 
| attended the Literacy Empowered training, Day 2. 86.2 25 13.8 4 29 
| attended the Literacy Empowered training, Day 3. 86.2 25 13.8 4 29 
| attended the Literacy Empowered training, Day 4. 82.8 24 17.2 5 29 
Total 29 
No Responses 7 
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Table A8. Efficacy and Instructional Practices of Literacy Empowered Survey Respondents in 


HISD, 2017-2018 


Strongly a Somewhat Strongly F 
Efficacy and Instructional Practices Disagree Disagiee Agree Agree Agree io peg 
% n % nn  % n % n % on % 9 
In comparison to one year ago, | havea 
deenes understanding’) how te 6.9 2 138 4 103 3 552 16 138 #4 ~ 29 3.55 


effectively incorporate literacy 
instruction into my content area. 

The digital and print reading materials 
and resources available to my classroom 6.9 2 3.5 1 20.7 6 48.3 14 20.7 6 29 3.72 
are high-quality. 

The digital and print reading materials 


and resources available to my classroom 10.3 3 6.9 2 20.7 6 44.8 13 17.2 5 29 3.52 
are useful. 

| facilitate the consumption of a high 

volume of texts including images, 3.5 1 3.5 1 31.0 9 37.9 11 24.1 7 29 3.76 


graphs, and charts. 
My students write for authentic purposes 


with varied content, audiences, and 10.3 3 3.5 a. 17.2 5 44.8 13 24.1 7 29 3.69 
modes. 

Students in my classroom independently 

select a "Just Right" book or text for 20.7 6 13.8 4 17.2 5 31.0 9 17.2 5 29 3.10 


independent reading. 
Students read books/various texts on 
their independent level during 


independent reading time daily in my 13.8 4 20.7 6 20.7 6 20.7 6 24.1 7 29 3.21 
classroom. 

l use the universal screener to form 

flexible reading groups. 31.0 9 13.8 4 24.1 7 27.6 8 3.5 1. 29 2.59 
l use the universal screener to assess 241 7 6.9 2 20.7 6 31.0 9 17.2 5 29 3.10 
student reading levels. 

l use the universal screener to monitor 

reading growth. 24.1 4 6.9 2 20.7 6 27.6 8 20.7 6 29 3.14 
Total 29 
No Responses 7 


Table A9. Respondents’ Perceptions of Their Literacy Empowered Instructional and Student 


Practices in HISD, 2017-2018 


Instructional and Student Strongly Disagree Somewhat Agree Strongly Total Weighted 
Practices Disagree Agree Agree Average 
% n % n % n % n % n n 


My students write for 
authentic purposes using 


varied content, audiences, 10.3 3 6.9 2 24.1 7 37.9 11 20.7 6 29 3.52 
and modes. 

My students are able to 

archive written work with 10.3 3 20.7 6 27.6 8 27.6 8 13.8 4 29 3.14 
portfolios. 

Py students analyze thelr 6.9 2 138 4 207 6 414 12 172 5 29 3.48 
work using rubrics. 

My students uulize witng 7A 2 71 #2 143 4 536 15 179 5 28 3.68 


as a tool for learning. 

l use open-ended questions 
and talk to facilitate 3.5 1 0.0 0 20.7 6 37.9 23. 37.9 11 29 4.07 
discourse in my classroom. 

| employ structured 

conversation that focuses 


on content and cognition in 3.5 1 3.5 i 13.8 4 48.3 14 31.0 9 29 4.00 
my instruction. 

In my classroom, dialogic 

discourse takes 

precedence over monologic 6.9 2 3.5 1 24.1 t 44.8 a3 20.7 6 29 3.69 
discourse. 

Total 29 
No Responses 7 
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Table A10. Survey Responses on Literacy Empowered Preparation and Support, HISD, 2017- 


2018 


Strongl . Somewhat Strongl , 
Preparation and Support pieanree Disagree Agree Agree Bisaniee Total Weighted 
% n % n % n % n % n n Average 
The information | learned during the 
Literacy Empowered summer training (s) 
prepared me for the implementation of 10.3 3 13.8 4 27.6 8 31.0 9 17.2 5 29 3.31 
Literacy Empowered in my classroom. 
The leaders on my campus provide 
valuable support and information about 
best literacy practices in the content sada ° ant f 2 : te at ans . = + 
areas. 
Total 29 
No Responses 7 


Table A11. Survey Reponses on LE and HISD Curriculum and Instructional Framework, 2017- 


2018 
Curriculum and Instructional strongly Disagree Sameniint Agree strongly NIA Total Weighted 
Framework Disagree Agree Agree Average 
% n % n % n % n % n % n n 
The structure and framework 
of the HISD secondary core 
content master courses 3.5 al 20.7 6 17.2 5 48.3 14 6.9 2 3.5 4. 29 3.36 
facilitate ease of planning. 
The instructional strategies 
outlined in the HISD 
secondary core content 
master courses provide 3.5 i 13.8 4 31.0 9 34.5 10 13.8 4 35 1 29 3.43 
support for my instructional 
planning and delivery. 
Total 29 
No Responses 7 
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Table A12. Summary of Survey Responses Related to Additional Training Required to 


Implement LE, HISD, 2017-2018 
# of 
Respondents 


Code Themes Responses 


e | would like to continue to receive best practices in Literacy 

BP More Best Practices 2 Empowered. 

e Hand on, not speech. 

e Class time to implement use of library books. 

e For campus administrators to allow us to use what was 
taught at the Literacy Empowered trainings instead of doing 
STAAR prep pushed on us by TDSs. 

e Actual planning time to incorporate it. 


TTl Time to Implement 4 


A follow-up from last summer. 

Support special education like SPED and ESL. 
Whatever is the next step. 

More literacy Empowered training. 

If we adopt new textbooks, a PD regarding the textbook 
would be helpful. 

e | would have appreciated clear directions during LE, last 

summer, stating THIS IS REQUIRED not, THIS IS 

9 SOMETHING THAT YOU CAN USE IN YOUR CLASSES. It 
would have made my school much easier instead of having 
to switch in midstream. For high school, we need a DAY on 
organizing and working in stations with classes OVER 30. 

e | need more training in implementing daily uninterrupted 

reading. 

e How to incorporate reading time/reading logs without 
straying from the math curriculum. Maybe a 
recommendation of books on math topics. 

More training in Language Live/Achieve 3000. 
More digital resources. 

NA, N/A 

n/a 

None 

nothing 

| don’t know. 

none at this time. 

The QTEL training was very good. 


MT More Training 


Res More Resources 


NA No Assessment/None 10 


AC Accolades 1 
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Table A13. Survey Responses on Improvement in HISD Master Courses to Support Literacy 


Empowered Implementation, 2017-2018 
Codes Theme # of Responses 
Respondents 


e | would like to continue to receive current strategies to utilize 
that are successful to further support literacy 
BP More Best Practices 2 implementation. 
e Literacy strategies incorporated in curriculum/reading 
resource to serve Tier |, Il and Ill students. 
e It’s hard to say because | am part of an Achieve 180 school, 
teaching three preps. It was hard to find time to go through 


mm More Tinie lG Iplcrient : the master lessons to see what they required and how to 
make adjustments for my scholars. 
e Something geared to the unique needs of Early College 
students. 
NSBT More Specific Needs- 3 e Realistic pedagogy in addressing multiple levels in a 


Based Training classroom including ELL, SPED, GT, and students with 
achievement gaps but who are not SpEd. 

e Student friendly language. 

e Ongoing training and part of the curriculum and lesson 
planning. 

e This is a hard question. Some of the lessons are formulaic 
and too short. They also seem repetitive, as if one person is 

TApp. Team Approach 1 writing them, using his or her own personal best strategies. It 

would be beneficial to have more people creating the 

lessons. 

| want more text variety 

Not sure 

NA (2) 

None (2) 

| have no clue at present 


MT More Training 


Res More Resources Needed 1 


NA No Assessment 6 
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Appendix B 
Literacy Empowered Initiative 
Lesson Walkthrough and Observations 
English | Classroom 
Date: May 17, 2018 Time: 9:45—-11:20 
The following objective as shown in Figure B1 was inscribed on the whiteboard: 


Lesson Objectives: The students will be able to utilize three big questions to guide thinking and improve 
comprehension of informational text. 


Do Now: Brain Dump - Elephants - (5 mins). 
Agenda: Discuss now — (5 mins.). 


1. Create three big questions chart (as a class) — (20 mins.). 
2. Create chart in group — (30 mins.). 


3. Read and answer question, “Can Elephants Jump?” — (30 mins.). 
Procedure: 
The teacher placed the three big questions on chalk board and on a flipchart. 


1. What surprised me? 
2. What did the author expect me to already know? 
3. What changed, challenged, or confirmed what | thought | knew? 


Figure B1. Overview of LE Lesson with Objective and Agenda. 
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Students were asked to rea to read the article, “Can Elephants Jump?” or they asked their peers to read. 
The Three-Big-Questions chart was completed as a class. Figure B2 presents the chart. 


Figure B2. Class Chart for Three-Big-Questions on “Can Elephants Jump?” 


Students were then asked to organize themselves into groups and complete the chart using the three big 
questions as their guides. Figure B3 shows the chart for one of the groups. 


Figure B3. Group Chart for the Three-Big-Questions on Elephants Observed During Literacy 
Empowered Walkthrough 


Observation: 


e The teacher clearly outlined the lesson objective, and the agenda or task students needed to undertake 
to accomplish the objective. 
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e The teacher demonstrated the task, working with the whole class, that students needed to accomplish, 
and how. 


e Students understood the task and were able to replicate the expectation and meet the objectives of the 
lesson working in groups. 


e The read aloud was a useful integration into the lesson to demonstrate fluency and one strategy for 
teaching comprehension. 


e It would have been ideal if students were allowed to read the article before the class to improve their 
comprehension through re-reading. The teacher could have used students’ prior knowledge of circus 
and circus animals to establish context and provide a more concrete discourse on the issue of animal 
cruelty, rights, and advocacy. 


During the walkthrough a sample of students’ writing portfolios were reviewed. Figure B4 shows the 
portfolios. 


Figure B4. Portfolios of Students’ Writing Observed During Literacy Empowered Walkthrough 


The review of the portfolios found that: 


e They were neatly shelved and contained in folders. Evidence of grading was present, but the grading 
rubrics and feedback were not visible. 


e The information to parents on assessment forms were unsigned. 


e Evidence of the use of the writing process that demonstrated progressive development from drafts to 
final publishable writing pieces was not apparent. 
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Lesson 2 
Literacy Empowered Initiative 
Lesson Walkthrough and Observations 
English 4 Classroom: Advanced Placement Literature and Composition 
Sustained Silent Reading 
Date: May 17, 2018 Time: 12:30 —1:30 
Lesson Objective: Demonstrate English comprehension and expand reading skills 


Lesson Procedure: 


e Students were organized into table groups and were asked, in those groups, to read two creation 
myths for comparing creation myths. 


e Students were asked to use anchor charts on wall and record information on orange butcher paper 
for 10—15mins. 


e The teacher had three sheets with the words, “Must Have,” “Might Have” and “Will Not Have” as 
shown in Figure B5 posted on the whiteboard to guide the assignment. Students were asked to 
volunteer answers to complete three blank anchor charts. 


Figure B5. Replica of Anchor Charts Displayed on Whiteboard 


Must Haves Might Have Will Not Have 
e Gods e Characters e Time 
e Creation e Settings e Different Gods (Why? Culture) 
e Beginnings e Message e Imaginations 
e Theme e Emotions e Different roles 
ie aiaeaieadiea acaearcs e Truth -reality (evidence-based??) 
e Myth-fake 
e Myths 
e Storyline 
e Messages 
e Characters 
e Middle 


Table task was assigned - 5 mins 
Required Action: 


e Create a creation myth — write your own myth 
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Figure B6. Guiding Questions for Class Activities on Creation Myths, 2017-2018 


Ww 7, : 
cr hdd Mel 

ine you rt of a trib 
world ~ Nasi 
sehen 


Walkthrough Observations 


Students may have required more thinking time to collaborate and create their own myths and a step- 
by-step approach over an extended period. 


There appeared to be no books or classroom library, “Saddle-back” or “Just Right” books. 


The teacher was unable to keep students’ attention. They were observed to be talking to each other 
while the teacher spoke, playing while the teacher was presenting the task. 


The teacher quizzed students about their perception and understanding of a myth. 


Students volunteered responses for the anchor charts rather than being called on to provide responses. 
This created uneasy lulls in the delivery. 


There were several other reading charts throughout the class: 
o Word gaps and quoted words, 
o Stop, notice, note charts, and 
o Ideas and organizations. 


Students appeared to be using several reading and comprehension strategies during the table task: 
o Annotation of information and reading, 
o Use of index cards (Some students), 
o Highlighting texts in the article, and 
o Discussion among peers. 
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Appendix C 


Table C1. Distribution of Students in the LE Sample by Key Demographic and Educational 


Attributes, 2017-2018 
Sample (9'" — 12" Grade) 


District (All Grades) 


Attribute Treatment Control 
n=10,277 n = 26,807 n = 214,175 
Gender Female 50.5 51.7 50.7 
Male 49.5 48.3 49.3 
Ethnicity Asian 2.5 5.4 4.1 
Black 31.3 24.3 24 
Hispanic 60.9 54.5 61.8 
White 4.3 14.0 8.7 
ESL No 76.7 88.3 31.5 
Yes 23.3 11.8 68.5 
Economically Disadvantaged No 15.6 31.7 24.1 
Yes 84.4 68.3 75.9 
At Risk No 20.9 36.3 28.4 
Yes 79.1 63.8 71.6 
Special Ed No 96.4 96.9 92.8 
Yes 3.6 3.1 te 
GIT Identification No 89.6 81.7 84.3 
Yes 10.4 14.3 15.7 


Source: HISD 2017-2018 PEIMS Access Data File; PEIMS. Note: Teffects ra used the exams data from this sample to determine 
the counterfactual or missing data to measure the LE effect. This method does not require a matched sample. 


Table C2. Percentage of HISD Literacy Empowered Student Sample by Course Who Performed 
At or Above STAAR EOC Approaches Grade Level Standard, 2017-2018 


Attribute n= 10,277 Algebra | Biology English | English II U.S. History 
n= 2,534 n= 2,420 n= 2,726 n= 2,168 n= 429 
Gender Female 
Ethnicity 
Hispanic 
ESL 


Economically 
Disadvantaged 
Special Ed 


At Risk 


GIT Identification 


Source: HISD 2018 STAAR EOC Access Data File. Green = 2 50 percent; Pink = < 50 percent met Approaches Grade Level 
standard 
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Table C3. Average Treatment Effects (ATE) of LE Initiative on 2018 STAAR Algebra I Exam 


Results, 2017-2018 
Robust [95% Confidence 


Scale Score erie dls Standard Error 2 gs Interval] 
ATE 
Treatment 
(1 vs 0) -23.7 -11.1 -2.2 0.000 -45.2 -2.3 
Potential Outcome Mean 
Treatment 
0 3932.0 7.1 556.7 0.000 3918.1 3945.8 


Table C4. Average Treatment Effects (ATE) of the LE Initiative on 2018 Students’ STAAR 


Biology Exam Results, 2017-2018 


Robust [95% Confidence 


Scale Score Coefficient Standard Error z P>z interval] 
ATE 
Treatment 
(1 vs 0) -95.0 12.3 -7.7 0.000 -119.1 -71.0 
Potential Outcome Mean 
Treatment 
0 4155.7 7.2 576.9 0.000 4141.5 4169.8 


Table C5. Average Treatment Effects (ATE) of the LE Initiative on 2018 Students’ STAAR English 


| Exam Results, 2017-2018 


Robust [95% Confidence 


Scale Score Coefficient Siandard ERD z P>z Interval] 
ATE 
Treatment 
(1 vs 0) -122.2 10.4 -11.8 0.000 -142.6 -101.9 
Potential Outcome Mean 
Treatment 
0 4045.5 6.8 595.6 0.000 4032.2 4058.8 


Table C6. Average Treatment Effects (ATE) of the LE Initiative on Students’ 2018 STAAR English 


ll Exam Results, 2017-2018 


Robust 


Scale Score Coefficient Standard z P>z sa Confidence 
nterval] 
Error 
ATE 
Treatment 
(1 vs 0) -48.0 10.0 -4.8 0.000 -67.6 -28.3 

Potential Outcome Mean 
Treatment 

0 4058.9 6.7 601.4 0.000 4045.7 4072.2 
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Table C7. Average Treatment Effects (ATE) of the LE Initiative on Students’ 2018 STAAR U.S. 


History Exam Results, 2017-2018 


Robust : 
Scale Score Coefficient main z P>z saat atains nes 
rror 
ATE 

Treatment 

(1 vs 0) -233.9 36.7 -6.4 0.000 -305.8 -161.9 
Potential Outcome Mean 
Treatment 

0 4311.1 7.9 544.1 0.000 4295.6 4326.6 


Table C8. Selected Predictors of LE Students’ Performance on the 2018 STAAR EOC Algebra | 


Exam, 2017-2018 


Scale Score Coefficient Beta Standard Error 

Special Education -312.3** -0.14 39.6 
GIT Identification 355.0** 0.19 34.4 
At Risk -275.7** -0.23 23.2 
Career and Technical Education 8.6 0.01 10.9 
Gender -65.7** -0.07 16.2 
Economically Disadvantaged -8.8 -0.01 24.5 
ESL -113.9** -0.12 19.1 
Constant 4148.2** 32.4 
R2 0.17 

F 75.9** 


**p, < .001 (two-tailed) 


Table C9. Selected Predictors of LE Students’ Performance on the 2018 STAAR EOC Biology 


Exam, 2017-2018 


Scale Score Coefficient Beta Standard Error 

Special Education -276.4** -0.12 40.6 
GIT Identification 334.4** 0.21 27.7 
At Risk -372.3** -0.32 21.3 
Career and Technical Education 15.2 0.02 10.4 
Gender 20.6 0.02 16.3 
Economically Disadvantaged -47.6 -0.03 23.7 
ESL -269.6** -0.22 21.8 
Constant 4283.4** 31.1 
R2 0.30 

F 152.3** 


**p. < .001 (two-tailed) 
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Table C10. Selected Predictors of LE Students’ Performance on the 2018 STAAR EOC English | 


Exam, 2017-2018 


Model | Model 2 
Stale Score Coefficient Beta Coefficient Beta Standard Error 

Special Education -341.2** -0.12 -181.9** -0.07 30.2 
GIT Identification 366.7** 0.22 213.2** 0.13 19.7 
At Risk -324.2** -0.26 -175.6** -0.14 15.2 
Career and Technical Education -11.2 -0.02 -5.9 -0.01 7.5 
Gender -129.2** -0.13 -32.4** -0.03 11.1 
Economically Disadvantaged -56.1 -0.04 -9.7 -0.01 16.8 
ESL -432.4** -0.37 -226.3** -0.19 14.2 
Writing Composition 208.0** 0.60 4.4 
Constant 4248.6** 3213.3** 31.0 
R2 0.42 0.70 

F 276.9%** 721.0** 


**p. < .001 (two-tailed) 


Table C11. Selected Predictors of LE Students’ Performance on the 2018 STAAR EOC English Il 


Exam, 2017-2018 


Model 1 Model 2 
Scale Score Coefficient Beta Coefficient Beta Standard Error 

Special Education -409.7** -0.11 -178.8** -0.05 38.3 
GIT Identification 424.5** 0.27 242.1** 0.15 18.7 
At Risk -380.0** -0.28 -236.7** -0.18 16.4 
Career and Technical Education -68.3** -0.09 -28.2** -0.04 7.6 
Gender -61.3** -0.05 -21.9 -0.02 12.1 
Economically Disadvantaged -118.3** -0.09 -48.3** -0.03 14.5 
ESL -575.3** -0.39 -238.4** -0.16 17.4 
Writing Composition 206.1** 0.58 4.7 
Constant 4477.6** 3266.7** 33.7 
R2 0.60 0.79 

F 471.3** 1024.9** 


**D. < .001 (two-tailed) 


Table C12. Selected Predictors of LE Students’ Performance on the 2018 STAAR EOC U.S. 


History Exam, 2017-2018 


ae Standard 

Scale Score Coefficient Beta Error 
Special Education -414.0* -0.14 118.6 
GIT Identification 460.1** 0.18 105.3 
At Risk -336.4** -0.23 63.4 
Career and Technical Education 28.7 0.05 24.3 
Gender 108.7 0.10 42.6 
Economically Disadvantaged 114.7 0.06 75.2 
ESL -417.9** -0.37 47.3 
Constant 4112.8** 93.2 
R2 0.3 
F 26.3** 


**p. < .001 (two-tailed), *p <.05 (two-tailed) 
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